<html>
<head><meta charset="utf-8"><title>vector of native width? · project-portable-simd · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/index.html">project-portable-simd</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html">vector of native width?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="239548009"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239548009" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mingye Wang <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239548009">(May 20 2021 at 07:40)</a>:</h4>
<p>Bad question: would it be beneficial to have some compile-time access to the native vector register width somehow? People writing some AoSoA data structure may benefit from that sort of stuff, but everyone writing their own bunch of <code>#[cfg]</code> to guess the size is probably not a good idea.</p>
<p>(And I assume that GPUs really need a <code>clinfo</code> call instead for that?)</p>



<a name="239549157"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239549157" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jacob Lifshay <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239549157">(May 20 2021 at 07:49)</a>:</h4>
<p>it may vary at runtime depending on the cpu and os, or even on how many registers the compiler decides to use (like in RISC-V V and SimpleV), so there may not be a good answer available at compile-time</p>



<a name="239550703"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239550703" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jubilee <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239550703">(May 20 2021 at 08:02)</a>:</h4>
<p><span class="user-mention silent" data-user-id="413916">Mingye Wang</span> <a href="#narrow/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F/near/239548009">said</a>:</p>
<blockquote>
<p>Bad question: would it be beneficial to have some compile-time access to the native vector register width somehow? </p>
</blockquote>
<p>As Jacob mentioned: There is not necessarily a coherent or good answer here.</p>



<a name="239551315"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239551315" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jubilee <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239551315">(May 20 2021 at 08:08)</a>:</h4>
<p>Trying to optimize around the "native register size" just on the assumption such will be more performant can wind up pessimizing performance. This applies for aiming for a 32-bit or 64-bit GPR, and it also applies for trying to mix things up based on 128-bit or 256-bit SIMD registers.</p>
<p>It is totally valid for an x86 microarchitecture to implement 64 physical 128-bit XMM registers and use those to implement the 16 xmm, ymm, and zmm architectural registers, using register renaming to deconflict, making it enormously faster at doing operations on individual xmm-wide registers since it doesn't have to use 4 at a time, even though it has access to what appears to be 512-bit registers.</p>
<p>This is simplified but not really hypothetical: Intel and AMD have both done something similar to this, repeatedly, over the years. And because using the full AVX512 width is more situational, with its license-based downclocking, LLVM likes to, even when AVX512F is enabled, use the narrower ymm and xmm registers, unless it knows it is targeting Ice Lake or KNL.</p>



<a name="239567056"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239567056" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Caleb Zulawski <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239567056">(May 20 2021 at 10:28)</a>:</h4>
<p>I think the way to do this (and the way I have done this in the past) is to make the vector width generic in your algorithm, and dispatch at runtime based on the instruction set, and change the generic parameter based on the instruction set.</p>



<a name="239636288"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239636288" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jubilee <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239636288">(May 20 2021 at 18:15)</a>:</h4>
<p>I would hope that the compiler just handles this at compile time, tbh, after you use whatever size of SimdArray you actually <strong>need</strong>.</p>



<a name="239646333"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239646333" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Caleb Zulawski <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239646333">(May 20 2021 at 19:29)</a>:</h4>
<p>I think that's optimistic because auto vectorization doesn't really work, but I mostly agree. Depends on algorithm</p>



<a name="239656834"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239656834" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jubilee <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239656834">(May 20 2021 at 20:46)</a>:</h4>
<p><span class="user-mention" data-user-id="312331">@Caleb Zulawski</span> I mean with our API that actually uses LLVM SIMD instructions.</p>



<a name="239657854"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239657854" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Caleb Zulawski <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239657854">(May 20 2021 at 20:55)</a>:</h4>
<p>I guess I meant that I think it's unavoidable to care about vector width to some extent, if the compiler is bad at auto vectorization I'm not sure it'll be smart enough to do something clever with your non-native vector sizes in all situations</p>



<a name="239657970"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239657970" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Caleb Zulawski <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239657970">(May 20 2021 at 20:56)</a>:</h4>
<p>Though I don't think that means we should provide a native width or anything (since there are so many factors that go into it), I just think developers will need to be aware of the width in some situations</p>



<a name="239663367"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239663367" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jubilee <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239663367">(May 20 2021 at 21:38)</a>:</h4>
<p>That's fair I guess. I think that with LLVM SIMD instructions we will be probably providing a strong enough "hint" that it <em>should</em> get it right, but I could still see problems emerging.</p>



<a name="239663812"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239663812" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jubilee <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239663812">(May 20 2021 at 21:41)</a>:</h4>
<p>There are many cases where LLVM can get an optimization correct if it just does another additional pass over the code, but it doesn't, because finite time, so it misses the optimization. Here, we're skipping past all the optimization passes just to recognize a valid vectorization site.</p>



<a name="239764930"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/257879-project-portable-simd/topic/vector%20of%20native%20width%3F/near/239764930" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alastair Reid <a href="https://rust-lang.github.io/zulip_archive/stream/257879-project-portable-simd/topic/vector.20of.20native.20width.3F.html#239764930">(May 21 2021 at 15:02)</a>:</h4>
<p>I think that Arm's SVE should have a set of intrinsics (that end up just reading a register) for accessing VL (the vector length).<br>
IIRC, these should come in several flavours</p>
<ul>
<li>Return VL/n: the number of elements of width &lt;n&gt; that fit in a vector (for n in 8, 16, 32, 64)</li>
<li>Return log2(VL/n): log of the number of elements of width &lt;n&gt; that fit in a vector. (I remember these being useful in DSP algorithms like FFT)</li>
<li>Return a mask which sets the first VL/n n-bit elements to active. (May be SVE specific since it would probably just be 'set all bits' in fixed-length architectures.)</li>
</ul>
<p>These sorts of intrinsics (or just constants on fixed-width architectures) are pretty useful for writing code that adapts to different vector lengths.<br>
Though, for this to work, you obviously don't want to use typenames that hardcode the vector length. So 'u8x16' is an inflexible name because it obviously only works for 128-bit vectors but vec_u8 (vector of u8) is meaningful across a range of vector widths.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>